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Summary, Findings, and Conclusions 

1. Since 1994 California State University, Los Angeles (Cal State LA) and California 
Polytechnic State University, San Luis Obispo (Cal Poly) have been using the mediated 
learning system (MLS) courseware for remedial mathematics developed by Academic 
Systems Corporation. Both campuses have undertaken various evaluation studies. 

2. Academic Systems' courseware was chosen at the time because it was the most 
sophisticated multimedia program available. It is highly interactive and provides 
several layers of student feedback. The MLS comes with a management system 
that captures student responses to questions and records time on task, thus allow- 
ing the instructor to monitor each student's progress. (The MLS courseware that 
was implemented runs in on-campus computer labs. A new version of the 
courseware is now accessible through the Internet.) 

3. By 1996 six additional CSU campuses at Hayward, Long Beach, Northridge, Pomona, 
Sacramento, and San Francisco had also begun implementation of the MLS courseware. 
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Benefits: Comparative Learning Outcomes 



4. Cal Poly has adopted the MLS for all of its remedial mathematics courses. The 
courseware is available in a 60-station computer lab where the student work is over- 
seen by graduate assistants under the direction of a faculty member. 

5. Cal State LA has adopted the MLS for the remedial coursework provided to the most 
mathematically challenged students. A three-year evaluation of the MLS at Cal State 
LA contains the following summary statements: 

(a) As a replacement for regular classroom instruction the MLS courseware has the 
most benefit for those students who need remediation the most, where they can 
take advantage of the flexibility to repeat a lesson as often as needed. 

(b) The MLS courseware is not superior for all remedial students nor is there any 
indication that it is inferior. 

(c) An instructor is still essential for the MLS version of the remedial courses both as 
a motivating influence and as a personal contact for the students. 

(d) From a resource perspective the MLS allows larger section sizes without a loss of 
student performance. 

(e) Other potential benefits include: more flexible use of student time compatible 
with more time on task, more instructional support outside of classroom time, 
completion of course on a shortened time schedule, more convenient and user 
friendly for ESL students, longer retention of the material, and availability of 
Internet access to MLS supporting distributed instruction to off-campus locations. 

6. An evaluation of learning outcomes in the remedial mathematics courses at the eight 
campuses was undertaken in fall 1996 at the request of the CSU Provosts/Vice Presidents 
for Academic Affairs. Over 4,500 students were enrolled in the courses; approximately 34 
percent were enrolled in the MLS sections, 66 percent in regular classroom sections. 

7. The MLS courseware was employed differently across the campuses. In some cases the 
MLS was used to supplement the regular weekly scheduled course meetings, in others 
it was an integral part of the course in the sense that its use replaced some, but not all, 
of the regular class meetings; at some sites, the MLS was used to essentially replace the 
regular class meetings and become the principal mode of providing the instruction. 
For all campuses, even if the entire course was scheduled to be delivered using the 
courseware, instructors would still call groups of students out of the lab for short 
"chalk talks" on particular topics as the need arose. 

8. Two levels of remedial mathematics courses were evaluated, elementary and intermediate. 
Learning outcomes are measured by course passing rates and the percent of students com- 
pleting the course with a final score of 70 or better. Comparisons of learning outcomes were 
made for the regular classroom (control) sections and the MLS (experimental) sections of 
the courses offered at each campus. 

9. The course passing rate in the elementary course favored the MLS sections over the 
classroom sections at all of the seven sites where control sections were offered. The 
difference was statistically significant at only one site. 
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10. The percent of elementary level students with a final score of 70 or better was in 
favor of the MLS sections at four of the seven sites; one of these differences was 
statistically significant. Two of the three differences in favor of the classroom 
sections were statistically significant. 

11. The course passing rate at the intermediate level was in favor of the MLS at four of six 
sites; one of these differences was statistically significant. Of the two sites where the 
difference favored classroom sections, one was statistically significant. 

12. The percent of students with a final score of 70 or better at the intermediate level was in 
favor of the MLS sections at three of the six sites; none of the differences were statisti- 
cally significant. Two of the three differences that favored the classroom sections were 
statistically significant. 

13. Based upon this evidence it is not possible to reach a definitive conclusion regarding 
the learning outcomes of MLS as compared to classroom instruction. 

(a) At Cal State LA there is evidence that MLS improves the passing rates for the most 
mathematically challenged students who can benefit from a substantial amount 
of drill and practice. 

(b) In the seven -campus study there is weak evidence that the MLS improves passing 
rates in the elementary courses, but there is no evidence that it improves passing 
rates in the intermediate course or that it increases the percentage of students 
receiving a final score of 70 or better at either level. To the extent MLS improves 
passing rates, and reduces drop out rates, campuses benefit by generating more 
FTE and related funding. 

(c) Other benefits that may accrue to MLS include providing students with an 
introduction to the use of the computer as a learning tool, a more user-friendly 
learning situation for ESL students, and the potential to deliver courses to 
off-campus sites. 



Cost Comparisons 

14. Certain basic patterns emerge from the comparison of the cost estimates for the two 

instructional modes: 

(a) At the lower levels of annual course enrollment, the costs of the MLS version of 
the course always exceed those of the classroom version. This occurs because of 
the fixed costs associated with the MLS course including the license fee and the 
costs of establishing and operating the initial MLS lab. 

(b) The incremental (or marginal) costs of additional enrollments in the MLS course 
tend to be less than those for the regular classroom course (assuming instructor 
pay rates for the different course sections are the same). This result occurs because 
the MLS fee revenue offsets some of the course's staffing cost. Incremental costs 
are also reduced to the extent average enrollment in the MLS sections is increased. 

(c) Once enrollment grows to the capacity of the MLS lab, an additional lab must be 
added causing a step-up in MLS costs. 
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15. At the current levels of campus enrollments in remedial mathematics courses (ranging 
between 300 and 1,500 students per year), the estimated direct costs of the MLS course 
exceed the costs of offering the instruction to the same number of students in the 
classroom course. For the case where MLS and classroom sections sizes are both equal 
to 30, a cost model developed based upon "typical" campus data shows a crossover 
(or breakeven) enrollment for the MLS exceeding 2,500 students per year. 

16. However, because the marginal costs of the MLS course tend to be less than those of 
the classroom version, with sufficient levels of annual enrollment and the appropri- 
ate combination of cost and fee factors, the costs of MLS can be less than the costs of 
classroom instruction. 

17. The two CSU campuses with the longest experience with the MLS have demonstrated 
cost savings by increasing section size without an increase in faculty workload (be- 
cause the MLS is the main source of instructional materials) and without a reduction 
in the quality of student learning outcomes. 
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Background and Context 

The California State University consists of twenty-two campuses located throughout the state. 1 
The campuses enroll over 344,000 students and offer degrees in over 200 subject areas. The 
CSU system includes urban institutions serving large, primarily local, inner-city populations 
as well as suburban and rural institutions drawing their enrollments from throughout the 
state. Undergraduate students are admitted who are in the upper one-third of the high school 
graduating class or who transfer as upper division students from community colleges. 

The CSU has programs for testing the proficiency of entering undergraduate students in English 
(started in 1977) and mathematics (1983). Students who fail the proficiency examinations are 
placed in remedial courses in the first term of their enrollment. Students who do not make 
adequate progress in their remedial work are expected to enroll elsewhere. The number of fresh- 
man students requiring remedial courses became a source of systemwide concern in the 1990s 
(typically over 50 percent of incoming freshmen fail the entry level mathematics examination). 
The CSU is currently operating under a Board of Trustees' resolution adopted in 1997 to reduce 
the percent of students requiring remedial work in mathematics as follows: to 42 percent by 
year 2001, to 26 percent by year 2004, to 10 percent by year 2007 (a similar provision is in place 
for English). 

This case study report is based upon the use of multimedia courseware developed by Academic 
Systems Corporation to provide instruction in remedial and developmental mathematics. Their 
Mediated Learning System courseware (MLS) was developed to provide multimedia instruc- 
tional materials in campus computer labs. 2 In fall 1994 two campuses, California State University, 
Los Angeles (Cal State LA) and California Polytechnic State University, San Luis Obispo (Cal Poly), 
began working with Academic Systems to implement the MLS as a component of their instruction 
for remedial mathematics. The intent was to exploit the potential of the computer to provide more 
flexibility for individual pacing, to better accommodate different learning styles, and to allow for 
increased time on task. It was also hoped that the introduction of more real life examples would 
improve student motivation to study elementary mathematics. Because many of the students at Cal 
State LA have English as a second language, there was also a need to identify ways to deal with the 
understanding of written mathematics. 

Academic Systems' courseware was chosen because it was the most sophisticated multimedia 
program available. It emphasizes use of video and sound animated text. It is highly inter- 
active and provides several layers of student feedback. The MLS comes equipped with a 
management system that captures student responses to questions and records time on 
task, thus allowing the instructor to monitor each student's progress and whether they 
are having any particular problems. 

Cal Poly and Cal State LA are quite different. Cal Poly is a polytechnic university with 
programs in agriculture, architecture, and engineering. Located on the central California 
coast, it draws its students from throughout the state. Cal State LA is a comprehensive 
urban university with large programs in business, teacher preparation, and engineering. It 
is located in east Los Angeles and draws its students primarily from the local region, which 
has a high proportion of Hispanic, African American, and Asian households. 



1 A twenty-third campus, CSU Channel Islands, is under development. 

2 After this study had started, Academic Systems announced that the courseware "AcademicOnline" 
will be available for wide area networks and the Internet. 
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The CSU Chancellor's Office was also interested in computer aided instruction and provided 
financial assistance to both Cal State LA and Cal Poly to implement the MLS program. Both 
campuses agreed to conduct studies of the MLS courseware. Both campuses had been using 
regular lecture classes to teach remedial mathematics and both campuses had senior faculty 
who were interested in experimenting with computer mediated learning. Academic Systems 
was very helpful and worked closely with each campus to implement the MLS. Beginning in 
1995, six other CSU campuses joined the experiment and reported varying degrees of success 
with the MLS as is discussed in more detail below. 



Hie Implementation of MLS Courseware for Remedial Mathematics at CSU Los Angeles 

The following narrative describes the evolution of the use of the MLS at Cal State LA. It was 
extracted from a paper by Marshall Cates, Professor of Mathematics at Cal State LA. :| The 
experience at Cal Poly was similar to that described below. 



1994-95 

"We started in fall 1994 with four sections of Introductory Algebra with a combined enrollment 
of 1 15 students. Our first concerns were: (1) how should the MLS be integrated with or used in 
our lecture classes and (2) how could we develop a consistent measure of student performance 
in the course. 

"Concern #1 was approached by assigning different teaching strategies to various course 
sections as follows: 

'bolt-on' - this involved the regular lecture classroom sessions in parallel with and in addition 
to the computer lab assignments; 

'50/50' - half of the class time was spent in the regular classroom and half was spent in the lab 
working on the desktop computers; 

'no lecture' - there were no regular classroom lectures, the instructor was available to answer 
questions and would occasionally arrange small group discussions; 

'no instructor' - this most severe alternative was offered to students who could not get into 
either the regular classroom version or one of the other experimental versions of the course. 
Students in the 'no instructor' version were offered the opportunity to try the material on 
their own, using as much computer lab time as they wished. They were told that no one was 
going to monitor their progress, but if they needed help that they could contact an assigned 
instructor who had volunteered time to work with them. 

"Concern #2 was answered by instituting a departmental final exam written by an instructor who 
was not teaching any of the classes. This exam was then team scored by all introductory algebra 
instructors. There were four experimental and eight control sections. Two instructors taught both 
experimental and control classes, which allowed for control of the instructor variable. 

"During this time we (the course instructors and the departmental coordinator) started to 
meet as a team every other week where we discussed each section's performance regarding 
time on task and progress as measured by the number of quizzes completed. All sections were 
performing at about the same level in regard to average quiz scores. 



3 Marshall Cates, A Case Study of the Use of Computers to Provide Remedial Instruction in Mathematics 
at California State University, Los Angeles, January 1998. 
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"Table 1 summarizes the results at the end of the first quarter in terms of: (a) the percent of 
students who completed the course (i.e., who took the final) as an indicator of persistence in 
the course, (b) average score on the final (perhaps the best summary measure at a given point 
in time of a student's learning of mathematics), (c) percent who passed the final of those who 
took it, and (d) percent of students who passed the course. The percentage of students passing 
the final is calculated based upon the number who completed the course. The percentage 
passing the course is calculated based upon total course enrollments. Even if there were no 
drops between census date and the final date, the two percentages could still differ because 
course passing depends upon midterms and homework as well as the final exam score. Except 
for the no instructor model, the overall results suggested that the MLS students were doing 
about as well as those in the regular classroom version of the course. We were not disadvan- 
taging the students nor were we out-pacing the traditional classroom model. Of course we 
didn't expect to make gains in our first trials. 



Table 1 -Student Performance in Remedial Math Course Sections, Fall 1994 



Type of 
instruction: 


No. of 
students 


Completed 

course 


Final 

score 


% who passed 
exam of those 
who took it 


Passed 

course 


Regular classroom 


268 


85% 


63% 


62% 


69% 


Bolt-on w/MLS 


31 


84% 


60% 


62% 


61% 


50/50 w/MLS 


36 


83% 


65% 


63% 


50% 


No lecture w/MLS 


30 


80% 


63% 


63% 


61% 


No instructor w/MLS 


18 


50% 


52% 


33% 


28% 


Average MLS* 




82% 


63% 


63% 


60% 



Excludes data for the ‘no instructor’ option. 



"By conducting student surveys we found that the no lecture and the bolt-on models were 
disliked by the students. In the bolt-on version, students felt that the extra lab time was a 
form of punishment. We tested the no instructor model three more times with some interest- 
ing results. In the fall of 1995, we offered a section of Intermediate Algebra with no instructor 
under the same conditions as before. We had a course completion rate of 30 percent with a 20 
percent pass rate. In winter quarter 1996 we provided the no instructor section with extensive 
support material and students received a telephone call every week. The completion rate 
increased to 56 percent with a pass rate of 47 percent. In spring quarter we repeated the 
winter quarter material, but there were no telephone calls. The completion rate declined to 
38 percent with a pass rate of 31 percent. Calling made a great deal of difference but not 
enough to put this option in competition with the others. 
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"During winter quarter, we offered one bolt-on section, one 50/50 lecture-lab section, one 
30/70 lecture-lab section, and one section of no lecture. The results are shown in Table 2 
where, again, student performance in the MLS sections compared favorably with that of the 
regular classroom sections. 



Table 2-Student Performance in Remedial Math Course Sections, Winter 1995 



Type of 
instruction: 


No. of 
students 


Completed 

course 


Final 

score 


% who passed 
exam of those 
who took it 


Passed 

course 


Regular classroom 


195 


80% 


63% 


60% 


65% 


Bolt-on 


28 


71% 


62% 


55% 


50% 


50/50 


25 


84% 


58% 


48% 


48% 


30/70 


30 


90% 


75% 


85% 


80% 


No lecture 


23 


78% 


58% 


44% 


74% 


Average MLS 




81% 


64% 


60% 


63% 



"In spring 1995 we experimented with reducing the amount of lecture, replacing it with 
more 'pull out' time. When a need was indicated by the management reports, each instructor 
was encouraged to pull the appropriate group of students off the computers into an attached 
conference room for a mini-lecture or student presented blackboard work. The no lecture 
model was losing favor. Both the instructors and the students wanted some instructor led 
discussion. We also found that with no lecture, students began to treat the class as an open 
lab, coming in late and leaving early. A ten-minute mini-lecture at the beginning and five- 
minute quiz at the end of each period were implemented to combat this tendency. Table 3 
shows the results. 
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Table 3-Student Performance in Remedial Math Course Sections, Spring 1995 



Type of 
instruction: 


No. of 
students 


Completed 

course 


Final 

score 


% who passed 
exam of those 
who took it 


Passed 

course 


Regular classroom 


178 


75% 


61% 


61% 


54% 


20/80 w/MLS 


19 


74% 


62% 


57% 


53% 


30/70 w/MLS 


21 


71% 


61% 


56% 


43% 


50/50 w/MLS 


31 


71% 


61% 


59% 


48% 


No lecture w/MLS 


12 


58% 


49% 


43% 


25% 


Average MLS 




71% 


60% 


56% 


45% 



"By the end of 1994-95 it appeared that student performance in the MLS sections was about 
equivalent to that in the lecture sections, we had done no harm. In addition, student surveys 
showed a high level of support for the MLS sections, with praise for the flexibility of working 
on Saturdays. The surveys also identified an unexpected benefit in terms of computer literacy. 
Many students commented to the effect that now they had taken a 'computer class' they 
would not be afraid to take another. We had never considered the course to be a 'computer 
class' but for many of these students the course apparently represented their first experience 
with actually operating a computer for a practical purpose over an extended period of time. 

"We also settled upon a 20 to 30 percent share of lecture time as our standard delivery mode. 
We had not found a superior way to use the mediated learning process, but instructors felt 
that we were making progress, and I felt that we hadn't tried all that we could. Luckily, I had 
the support of the senior administrators who had never demanded that I show immediate 
results, but who were always interested in what we were doing. 



1995-96 

"In the fall of 1995 additional MLS courseware became available for Intermediate Algebra. 
This allowed computer coverage of all of our remedial mathematics courses. Three strands of 
remediation were now offered. Introductory Algebra, Intermediate Algebra, and Intensive 
Learning Experience (ILE - this consists of a one-year course that covers both Introductory 
and Intermediate Algebra for students who scored in the lowest quartile of the entry level 
mathematics examination). 

"We formalized some of our studies and introduced a pre-test so that we could measure 
growth and control for the variable of initial student preparation. As Table 4 shows, the 
MLS sections were superior to the regular classroom method for the ILE sections, our lowest 
level of mathematics. 
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Table 4-Student Performance in Three Levels of Remedial Math Sections, 1995-96 



Type of 
instruction: 


No. of 
students 


Completed 

course 


Final 

score 


% who passed 
exam of those 
who took it 


Passed 

course 


ILE - regular 
classroom 


379 


83% 


53% 


39% 


47% 


ILE - MLS 


127 


86% 


51% 


44% 


54% 


Intro. Alg. - 
regular classroom 


937 


81% 


65% 


57% 


61% 


Intro. Alg. - MLS 


171 


80% 


65% 


59% 


53% 


Intermed. Alg. - 
regular classroom 


709 


79% 


60% 


57% 


57% 


Intermed. Alg. - MLS 


153 


81% 


66% 


59% 


57% 



"The MLS courseware as implemented at Cal State LA worked best at the lowest level (ILE), for 
students who needed to review material over and over again, and at the highest level (Inter- 
mediate Algebra), where students could take advantage of the option to speed ahead when all 
that they needed was to review a section . 4 Based upon final exam scores, there did not seem 
to be an advantage for the Introductory Algebra course. (The difference in the pass rates for 
Introductory Algebra was statistically significant and favored the regular classroom version of 
the course.) 

"Shifting the ILE sections to MLS offered the ability to double average section size (because 
the instructors could rely upon the courseware to provide the bulk of the actual instruction) 
while keeping performance equivalent, at least. This saving, in terms of course staffing costs, 
essentially offset the cost of the MLS license. Shifting more students in Intermediate Algebra 
to the MLS format offered no cost advantage as average section size remained the same. 

"In order to replace graduating faculty (graduate teaching associates) and to train new graduate 
students who had never taught before, we started a process of team teaching classes in the 
fall. New instructors were teamed with experienced instructors for the fall quarter. In subsequent 
quarters, these new faculty began teaching their own sections of the course. 



4 The differences in the ILE and Intermediate Algebra pass rates were not statistically significant 
at the 5 percent level, however. 
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1996-97 



"During 1996-1997 we continued the team teaching and we continued to use MLS for some 
of the introductory and intermediate sections. During the year we experimented to see if 
we could use the courseware to allow students to proceed at different paces. Based upon 
previous experience we had an indication that 'self-pacing' in effect resulted in 'no pacing,' 
but we decided to try again. Eleven students were allowed to continue into the summer to 
finish their one-year sequence. I consider this experiment to have been a failure. Several 
students took the opportunity to take more time but this had no effect upon overall course 
completion rates. 

"Beginning in fall 1996 we adopted the MLS in all the ILE sections. Table 5 compares the 
experience of all the classroom ILE sections in 1995-96 with the MLS sections in 1996-97. 
Since ILE is a one-year, three-course sequence, the completion rate is based upon the enroll- 
ment in the first course (Math 81) and the number of students who completed the third 
course (Math 83). The difference in the completion rates shown in Table 5 is in favor of the 
MLS courseware and is statistically significant at the 1 percent level. Given that the regular 
classroom sections were designed for a class size of 15 while the MLS sections were designed 
for 30, the MLS approach was clearly successful. 



Table 5-Performance of ILE Students in Regular Classroom Sections (1995-96) and MLS 
Sections (1996-97) 



Type of remediation: 


Students 
starting Math 81 


Students 
successfully 
completing 
the sequence 


Percent 
completing 
the sequence 


1995-96 

Regular classroom 
Math 81-82-83 


187 


38 


14.6% 


1996-97 

MLS 

Math 81-82-83 


224 


89 


39.7% 



Follow-up Studies 

"Direct comparisons based upon test scores do not reveal the whole picture. We also studied 
what happens to students after they leave the remedial program. We conducted two studies. 

"The first study, summarized in Table 6, was a 'forward tracking study' that tracked all remedial 
students who had taken Elementary Algebra in fall 1994. We counted the number of quarters 
that had elapsed before these students completed the general education course (equivalent to 
College Algebra), the completion rate after 5 quarters (through spring quarter 1996), and 
average GPA in the general education course. Fall 1994 was our first quarter for using the MLS 
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and, therefore, we were least experienced with it. At the time the study was done, however, 
no later quarter would have allowed enough time to obtain a sufficiently large proportion of 
students who had completed the general education course. The MLS students in the sample 
had a slightly lower completion rate (the difference between the two rates was not significant 
at the 5 percent level), a slightly shorter time to completion, and a slightly lower GPA. 



Table 6-Performance of Remedial Students in General Education Course 



Remedial students who 


Regular 




took Elem. Alg. fall 1994 


classroom 


MLS 


383 


268 


115 


No. of students completing 


101 


39 


general education 


(38%) 


(34%) 


Average quarters to complete 
general education 


3.82 


3.74 


Average GPA in general ed. 


2.53 


2.38 



"The second study involved tracking all enrollments in the general education course in winter 
quarter 1997 backwards to see how each student had qualified for the course. This 'backward 
tracking study' had two advantages. First, we are able to make comparisons with students 
who didn't need remediation and, second, we were not dealing with a truncated population. 
In the forward tracking study shown in Table 6, more than half of the students in the initial 
remedial population are not available for the follow-up comparisons, whereas in the backward 
tracking study every student in the initial population is accounted for in the comparison 
population. Table 7 shows the categories of students by type of remediation, if any, and their 
average GPAs in the general education course. The count of remedial students in the table 
includes those who needed only one remedial course (as distinct from the 1LE students who 
needed three courses). Two results are suggested by the table: students who required a remedial 
course do not do as well in the general education course as students who did not require a 
remedial course, and the remedial students who took the MLS sections performed slightly 
better than those who took the regular classroom sections. 
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Table 7-Performance of All Students in the General Education Course, Winter 1997 



Type of remedial 


Number of students 


Average GPA in the 
general education 


instruction: 




course 


Regular classroom 


353 


1.87 


MLS 


29 


1.94 


None 


633 


2.39 



"We are currently starting to use a new version of the MLS courseware that provides access 
over the Internet. Initially our goal is to reduce the amount of open lab access time that must 
be scheduled and replace it with general lab access and home access. Eventually we hope to 
evolve a program that will reduce the amount of time a student needs to spend on campus. 
There are several obstacles. First and foremost is student study habits that are overly reliant 
upon the rhythm of two or three weekly class meetings. Second is the need to provide students 
with convenient access for interaction with an instructor from a distance. 



Summary of Experience with Implementation at Cal State LA 

• "As a replacement for regular classroom instruction the MLS courseware has the most 
benefit for those students who need remediation the most, where they can take 
advantage of the flexibility to repeat a lesson as often as needed. 

• The MLS courseware is not superior for all remedial students nor is there any indication 
that it is inferior. 

• An instructor is still essential for the MLS version of the remedial courses both as a 
motivating influence and as a personal contact for the students. 

• From a resource perspective the MLS allows larger section sizes without a loss of student 
performance. 

• Other potential benefits include: more flexible use of student time compatible with 
more time on task, more instructional support outside of classroom time, completion 
of course on a shortened time schedule, more convenient and user friendly for ESL 
students, longer retention of the material, and availability of MLS on the Internet 
supports distributed instruction to off-campus locations." 
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Benefits of MLS - Student Learning Outcomes 

This section on learning outcomes is a summary of the reports from the eight CSU campuses 
participating in the study to evaluate Academic Systems Corporation's interactive mathematics 
courseware materials, the Mediated Learning System (MLS) during fall term 1996. 

The CSU ad hoc committee, appointed to oversee the evaluation effort, met and agreed to 
report a common set of data elements for evaluating the impact of the MLS courseware on 
learning outcomes in the introductory and intermediate algebra courses. The data included 
an entry level measure of mathematical skills (the MDTP test) and exit level performance as 
measured by final exam scores, course grades, and course completion rates. 

Because some of the campuses were just starting to use the MLS while others had 2-3 years of 
experience, it was decided not to identify campuses by name. For purposes of this report the 
campuses are labeled sites A through H. The number of course sections and total enrollments 
for both the MLS and classroom versions of the elementary and intermediate courses are 
shown in Table 8. At the elementary level, Site B had no regular classroom (control) type 
sections, Site C had 14 classroom compared to eight MLS (experimental), and Site F had two 
MLS sections compared to 17 classroom sections. At the intermediate level, Site A had three 
MLS sections and 13 classroom sections, Site B, again, had no classroom sections, Site C had 
no intermediate sections at all, and Site F had two MLS and 27 classroom sections. In total, 
almost 4,700 students were enrolled in 155 sections of elementary and intermediate remedial 
mathematics. Approximately 34 percent of the enrollments were in MLS sections, 66 percent 
in regular classroom sections. 



Learning Outcomes, Results at Eight Campuses 

The MLS courseware was employed differently across the campuses. Some of the campuses 
had been using the MLS for several terms, some had just implemented it. Based upon the 
experience of the campuses that have been using the courseware for several terms, there is a 
learning period during which the course configuration is likely to be changed somewhat in 
response to the particular campus needs. One of the campuses that has used the MLS for 
several terms has adopted it for all of its remedial mathematics instruction. 

In some cases the MLS was used entirely as a supplement to the regular weekly scheduled 
course meetings, in other cases it was used as an integral part of the course in the sense that 
its use replaced some, but not all, of the regular class meetings; at some sites, the MLS was 
used to essentially replace the regular class meetings and become the principal mode of 
providing the instruction. These different uses are summarized in the campus sections below 
where the amount of regular classroom time that was displaced by the courseware is estimated. 
For all campuses, even if the entire course was scheduled to be delivered using the courseware, 
instructors would still call groups of students out of the lab for short "chalk talks" on particular 
topics as the need arose. In most cases students were expected to spend time, in addition to the 
regularly scheduled class hours, in the MLS labs doing homework or otherwise using the courseware. 




CSU Case Study 



17 



19 



Table 8-Number of MLS and Classroom Sections and 
Fall 1996 



Enrollments by Site and Course Level, 







MLS Sections 


Classroom Sections 


Sites/course levels 


Sections 


Students 


Sections 


Students 


Site A 


elementary 


3 


88 


4 


101 




intermediate 


3 


80 


13 


337 


Site B 


elementary 


5 


265 


0 


0 




intermediate 


5 


253 


0 


0 


Site C 


elementary 


8 


225 


14 


475 




intermediate 


0 


0 


0 


0 


Site D 


elementary 


3 


90 


3 


129 




intermediate 


4 


118 


5 


210 


Site E 


elementary 


4 


70 


5 


75 




intermediate 


5 


83 


4 


67 


Site F 


elementary 


2 


46 


17 


415 




intermediate 


2 


55 


27 


931 


Site G 


elementary 


3 


80 


3 


95 




intermediate 


3 


64 


3 


118 


Site H 


elementary 


2 


47 


2 


81 




intermediate 


2 


50 


1 


39 


Total 


elementary 


30 


911 


48 


1,371 




intermediate 


24 


703 


53 


1,702 


Total 


all levels 


54 


1,614 


101 


3,073 




Total enrollments 




4,687 






Total sections 




155 





In addition to the number of course sections and census date enrollments, each site reported 
scores on the CSU-UC Mathematics Development Project Test (MDPT), which was used as a 
pre-test for students enrolled in both elementary and intermediate sections of the courses. 
Four outcome results were reported in the original evaluation study: 5 the number of enrollees 
who took the final (a measure of attrition), average final exam scores, number of enrollees 
passing the course, and the number who received a final score of 70 or better. The site specific 
data reported in Tables 9A through 9H below include: average MDPT scores, average section 
size, total enrollment in the experimental and control sections, percent of students passing 
the course, and percent taking the final who received a score of 70 or better. 



s See J. R. Frankel, Academic Systems Corporation Interactive Mathematics Evaluation of Learning 
Outcomes, San Francisco State University, June 1997 . 
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Site A 



Approximately 75 percent of the course work was based upon the MLS courseware at this 
campus. All sections were taught from a common syllabus. Course pacing was standardized. 
All students took instructor-designed, coordinator-approved weekly quizzes and two midterms. 
Daily homework was required and graded. The regular classroom/lecture (control) sections 
were scheduled for four hours of lecture/discussion per week with some minimal use of small 
groups at the discretion of the instructor. The MLS (experimental) sections had one hour of 
lecture per week in a classroom and three hours of class time on-line supplemented as necessary 
with one-on-one or small group instruction. 

Elementary - In terms of the MDPT pre-test scores the two groups were essentially equal. Eighty 
percent of the MLS students successfully completed the course compared to 71 percent in the 
regular classroom sections, a differential pass rate for these groups of over 12 percent. 6 The 
MLS group also had a higher rate, by about 1 1 percent, of passing the course with final scores 
of 70 or better. Neither of these differences was statistically significant from zero at the 5 
percent level (i.e., neither of the "z" values was greater than 1.96). 

Intermediate-The pass rate for the MLS sections was over 7 percent higher than for the classroom 
sections but the difference was not statistically significant. The lecture sections had a 
substantially higher proportion of final scores of 70 or better (negative percentages shown in 
the table indicate differences in favor of the classroom sections), and, in this case, the difference 
was statistically significant and in favor of the classroom group. 



Table 9A-Comparison of Student Performance in MLS and Lecture Sections 







Regular 


Percent 


Difference 


Course level 


MLS 


classroom 


difference 7 


significant 
@ 5%? (“z”) 


Elementary 

Avg. MDPT score 


70 


69 






Avg. section size 


29 


25 






Total enrollment 


88 


101 






Percent passing 


80% 


71% 


+12.7% 


No (1.31) 


Percent passing w/ 


60% 


54% 


+11.1% 


No (0.89) 


score of 70 or better 








Intermediate 










Avg. MDPT score 


57 


59 






Avg. section size 


27 


26 






Total enrollment 


80 


337 






Percent passing 


59% 


55% 


+7.3% 


No (0.53) 


Percent passing w/ 


15% 


30% 


-50.0% 


Yes (2.74) 


score of 70 or better 











6 The difference between the two rates is 9 percentage points (80-71). The relative difference, 
using the classroom rate as the base, is 12.7 percent (9/71 = 0.127). 

7 The percent difference in this series of tables is calculated as discussed in the previous footnote. 
A positive percent difference indicates the MLS value is greater, a negative difference indicates 
the classroom value is greater. 
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Site B 



The MLS courseware constituted approximately 95 percent of the course work at this campus. 
The campus has used the MLS materials for several terms and has adopted it for all remedial 
mathematics instruction in its two courses. All courses are three quarter units and are taught 
by graduate assistants under the supervision of a part-time faculty member. All instruction 
occurs in a large (60-station) computer lab constructed for the courses. Except for group 
orientation, occasional small group sessions and explanations, and one-on-one or small-group 
sidebars, all work was done on-line. 

The course pass rates and percent passing with final scores of 70 or better for this site compare 
very favorably with those at the other campuses. It should also be noted that the average 
MDPT scores at this site are in the upper range of those observed at other sites. 



Table 9B-Student Performance in MLS Sections 



Course level 


MLS 


Elementary 




Avg. MDPT score 


71 


Avg. section size 


53 


Total enrollment 


265 


Percent passing 


73% 


Percent passing w/ 


73% 


score of 70 or better 




Intermediate 




Avg. MDPT score 


57 


Avg. section size 


51 


Total enrollment 


253 


Percent passing 


74% 


Percent passing w/ 


79%* 


score of 70 or better 





* “Percent passing with a score of 70 or better” is calculated based upon number of students who 
took the final. This value can exceed “percent passing,” which is based upon total students enrolled. 
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SiteC 



A four-quarter unit elementary course was offered; there was no intermediate course. All of the 
course work in the MLS sections was based upon the MLS courseware. Lecture section students 
spend four hours in lecture/discussion classes. All instruction for MLS sections takes place in 
the computer lab where students receive approximately one hour of lecture/discussion and 
three hours on-line instruction per week. 

Students in the lecture sections had a higher average score on the MDPT test but the MLS 
students passed the course at a 54 percent higher rate. The MLS students also did better on 
the final scores, earning grades of 70-plus at a 46 percent higher rate. Both differences were 
statistically significant. 

Table 9C-Comparison of Student Performance in MLS and Lecture Sections 







Regular 


Percent 


Difference 


Course level 


MLS 


classroom 


difference 


significant 
@ 5%? (“z”) 


Elementary 

Avg. MDPT score 


43 


55 






Avg. section size 


28 


34 






Total enrollment 


225 


475 






Percent passing 


77% 


50% 


+54.0% 


Yes (6.81) 


Percent passing w/ 


51% 


35% 


+45.7% 


Yes (4.19) 


score of 70 or better 
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Site D 



All courses were three semester units. Lecture sections met in regular classrooms. The MLS 
courseware constituted approximately 95 percent of the MLS version of the courses. The MLS 
sections met in a computer laboratory and were assisted by instructors primarily on a one-on-one 
basis. All MLS section work was done on-line. All students took a common final examination. 

The students in the intermediate MLS sections had higher MDPT scores. The passing rate and 
the percent passing with a score of 70 or better were higher for the MLS students in both the 
elementary and intermediate courses. Although the differences in all the ratios favored the 
MLS sections, none of the differences were statistically significant. 

Table 9P-Compar8son of Student Performance in MLS and Lecture Sections 



Course level 


MLS 


Regular 

classroom 


Percent 

difference 


Difference 
significant 
@ 5%? (“z”) 


Elementary 










Avg. MDPT score 


68 


67 






Avg. section size 


30 


43 






Total enrollment 


90 


129 






Percent passing 


86% 


81% 


+6.2% 


No (0.95) 


Percent passing w/ 


79% 


69% 


+14.5% 


No (1.60) 


score of 70 or better 










Intermediate 










Avg. MDPT score 


52 


46 






Avg. section size 


30 


42 






Total enrollment 


118 


210 






Percent passing 


70% 


60% 


+16.7% 


No (1.95) 


Percent passing w/ 


46% 


40% 


+15.0% 


No (1.12) 


score of 70 or better 
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Site E 



Both the classroom and MLS sections met four times per week for four units. The curriculum 
for both was paced and required in-class examinations. The MLS courseware constituted 
approximately 95 percent of the content in the MLS sections. The classroom sections enrolled 
15-17 students and were taught as tutorials by three student assistants who were supervised 
by a part-time faculty member. The MLS sections enrolled 17-18 students and were taught in 
a 20-station computer laboratory by two graduate assistants supervised by a part-time faculty 
member. All work in the MLS sections was done on-line. 

The MLS students in the elementary course had a slightly higher average MDPT score and course 
passing rate, but an 1 1 percent lower pass rate with final scores of 70 or better; neither of the latter 
two differences was statistically significant. The MLS students in the intermediate sections had 
slightly lower MDPT scores and a course passing rate 21 percent lower than the lecture students. 
This difference was statistically significant. The MLS students also had a higher rate of passing 
with final scores of 70 or better but the difference was not statistically significant. 



Table 9E-Comparison of Student Performance in MLS and Lecture Sections 







Regular 


Percent 


Difference 


Course level 


MLS 


classroom 


difference 


significant 
@ 5%? (“z”) 


Elementary 

Avg. MDPT score 


71 


67 






Avg. section size 


18 


15 






Total enrollment 


70 


75 






Percent passing 


86% 


83% 


+3.6% 


No (0.50) 


Percent passing w/ 


58% 


65% 


-10.8% 


No (0.87) 


score of 70 or better 










Intermediate 










Avg. MDPT score 


48 


52 






Avg. section size 


17 


17 






Total enrollment 


83 


67 






Percent passing 


65% 


82% 


-20.7% 


Yes (2.32) 


Percent passing w/ 


31% 


30% 


+3.3% 


No (0.19) 


score of 70 or better 
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SiteF 



All courses were three semester units. The classroom sections were scheduled three hours 
per week in a regular classroom. Use of the MLS courseware accounted for approximately 
67 percent of the course work in the MLS sections. The MLS sections were scheduled one 
hour per week in a classroom and two hours in the computer lab. 

The MLS students at both levels had slightly lower average MDPT scores, higher course passing 
rates, and a higher percent passing with final score of 70 or better. None of the differences were 
statistically significant. 



Table 9F-Comparison of Student Performance in MLS and Lecture Sections 







Regular 


Percent 


Difference 


Course level 


MLS 


classroom 


difference 


significant 
@ 5%? (“z”) 


Elementary 

Avg. MDPT score 


51 


53 






Avg. section size 


23 


24 






Total enrollment 


46 


415 






Percent passing 


74% 


62% 


+19.4% 


No (1.54) 


Percent passing w/ 


64% 


55% 


+16.4% 


No (1.17) 


score of 70 or better 










Intermediate 










Avg. MDPT score 


40 


42 






Avg. section size 


28 


34 






Total enrollment 


55 


931 






Percent passing 


71% 


64% 


+10.9% 


No (1.08) 


Percent passing w/ 


64% 


55% 


+16.4% 


No (1.31) 


score of 70 or better 
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Site G 



All courses were four quarter units. The MLS courseware constituted approximately 95 of 
the content of the MLS sections. The students in the MLS sections spent four hours per 
week in a lab setting, students in the classroom version of the courses spent four hours per 
week in lecture discussion sessions. Students in the MLS sections were also expected to 
spend (an unspecified amount of) time in the computer lab. 

At both course levels the MLS sections had lower average section enrollments, slightly higher 
MDPT scores, and higher course passing rates than the lecture sections. At both levels the 
lecture sections had a substantially higher passing rate with final scores of 70 or better. The 
difference in the passing rate for the elementary sections was not statistically significant. The 
difference in the passing rate at the intermediate level and the passing rate with a final score 
of 70 or more at both levels was statistically significant. 

Table 9G-Comparison of Student Performance in MLS and Lecture Sections 



Course level 


MLS 


Regular 

classroom 


Percent 

difference 


Difference 
significant 
@ 5%? (“z”) 


Elementary 










Avg. MDPT score 


69 


64 






Avg. section size 


27 


32 






Total enrollment 


80 


95 






Percent passing 


73% 


60% 


+21 .7% 


No (1.74) 


Percent passing w/ 


36% 


58% 


-37.9% 


Yes (2.88) 


score of 70 or better 










Intermediate 










Avg. MDPT score 


45 


41 






Avg. section size 


21 


39 






Total enrollment 


64 


118 






Percent passing 


68% 


53% 


+28.3% 


Yes (2.01) 


Percent passing w/ 


28% 


53% 


-52.8% 


Yes (3.30) 


score of 70 or better 
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Site H 



The MLS courseware constituted approximately 90 percent of the work in the MLS sections. 
All sections were for five semester units that required students to attend five hours per week. 
All sections required interactive learning. The non-MLS sections (control sections) relied upon 
collaborative learning/group problem solving approaches in combination with 10-15 minute 
instructor-led discussions with the entire class. 

The elementary control sections enrolled an average of 41 students in a course taught by 
a graduate teaching associate assisted by two student tutors. The MLS sections enrolled an 
average of 24 students, all of whom were repeating the course. Topics not included in the 
MLS courseware were taught in classrooms the same way as in the control sections. Most 
of the MLS instruction occurred in the computer lab with occasional groupwork in an 
adjoining classroom. 

The elementary MLS sections had substantially smaller average enrollments than the control 
sections. There was no difference between the MDPT scores. The MLS section had an 11 
percent higher passing rate, but the control section had an 81 percent higher rate for students 
passing with a final score of 70 or better. This latter difference was statistically significant. 
The intermediate MLS sections had smaller average enrollments and lower average MDPT 
scores than the classroom sections. Both the pass rate and the percent with final score of 70 
or better were substantially higher for the control sections but the differences were not 
statistically significant. 

Table 9H-Comparison of Student Performance in MLS and Lecture Sections 







Regular 


Percent 


Difference 


Course level 


MLS 


classroom 


difference 


significant 
@ 5%? (“z”) 


Elementary 

Avg. MDPT score 


35 


35 






Avg. section size 


24 


41 






Total enrollment 


47 


81 






Percent passing 


40% 


36% 


+11.1% 


No (0.52) 


Percent passing w/ 


5% 


27% 


-81 .5% 


Yes (3.09) 


score of 70 or better 










Intermediate 










Avg. MDPT score 


48 


53 






Avg. section size 


25 


39 






Total enrollment 


50 


39 






Percent passing 


38% 


54% 


-29.6% 


No (1 .49) 


Percent passing w/ 


12% 


22% 


-45.5% 


No (1.24) 


score of 70 or better 
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Student Performance Across Sites 



Chart I provides a comparison of the course passing rates for the seven campuses that had 
both classroom and MLS versions of the elementary course. The data have been sorted by 
classroom passing rates in ascending order to facilitate the visual display. In all seven cases 
the MLS passing rate exceeds the classroom passing rate. The difference was statistically 
significant only for site C. 



Chart 1-Passing Rates: Elementary 
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Chart 11 provides a comparison of the percent with a final score of 70 or better for the elementary 
course. Here the MLS sections did better than the classroom sections at four of the seven sites; 
the difference was statistically significant only at site C. Of the three sites where the differences 
favored classroom sections, those at sites H and G were statistically significant. 



Chart 11-Percent w/Final Score of 70 or Better: Elementary 
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Charts III and IV provide data similar to Charts I and II for the six campuses that offered 
classroom and MLS sections of the intermediate course. In terms of course passing rates, 
as shown in Chart III, the MLS students did better at four of the six sites. The difference 
was significant (in favor of the MLS sections) only at site G. At the two sites where the 
passing rates favored the classroom students, the difference was statistically significant at 
site E. 

In terms of passing rates with final scores of 70 or better, as shown in Chart IV, the MLS 
sections did better at three of the six sites but none of the differences were statistically 
significant. At the three sites where the passing rates with final scores of 70 or better favored 
classroom sections, the differences were significant at sites A and G. 



Chart Ill-Passing Rates: Intermediate 
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Chart IV-Percent w/Final Score of 70 or Better: Intermediate 
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Charts V and VI show the passing rate data associated with average MDPT scores. The hypothesis 
is that students with higher scores on the MDPT pre-test should have a higher probability of 
passing the course without regard to whether they were in a classroom or MLS section. The scatter 
of points representing the average MDPT scores and average passing rates should show a trend 
from the lower left to the upper right of the chart. To the extent the MLS materials systematically 
improve course passing rates, the MLS points should lie above the classroom points. 

Both of these patterns are present for the elementary course as shown in Chart V. The positive 
correlation between MDPT scores and passing rates is most obvious for the classroom sections. 
This same pattern is less clear for the MLS sections (it is even less obvious if the single MLS point 
on the lower left of the chart is ignored). There is a tendency for the MLS passing rates to lie 
above the classroom passing rates, but the tendency is much stronger for the sections with low 
average MDPT scores. The implication is that the MLS is most beneficial to those students in 
the elementary course who are most challenged mathematically. 

The patterns for passing rates in the intermediate course shown in Chart VI are more 
ambiguous. There is a very slight tendency for the MLS rates to lie above the classroom 
rates for given MDPT scores, but the pattern is contradicted by two outliers, a classroom 
passing rate above all others and an MLS passing rate below all others. 



Chart V-Pass Rates vs. MDPT Scores: Elementary 
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Chart Vi-Pass Rates vs. MDPT Scores: Intermediate 
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Costs of MLS Compared to Regular Classroom Costs 

For purposes of developing campus cost estimates for MLS and regular classroom instruction 
the ad hoc committee agreed to report a set of data relating to the costs of the computer lab 
where the MLS courseware is available and the costs of staffing both the MLS and the lecture 
course. The results discussed below are based upon data provided by six campuses. 

The MLS courseware has been implemented in different ways at different campuses. These 
differences lead to different costs. This is especially true in terms of laboratory monitoring 
costs and in terms of the fees charged students for the MLS workbooks (fee revenue is treated 
here as an offset against the costs associated with the MLS course). Section enrollments in the 
MLS version of the course varied between 1 5 and 50. In some cases enrollments were restricted 
because of existing computer lab capacity. In others, large labs had been specifically designed 
to allow the courseware to serve as the basis for a substantial amount of the actual coursework 
with instructors primarily responsible for monitoring student progress and intervening only 
as necessary. 8 

Costs of the MLS courses were estimated based upon three versions of a basic model that 
relates costs to annual course enrollments. The cost parameters of the model shown in Tables 
10, 11, and 12 are based upon data supplied by the campuses. The basic cost model combines 
characteristics of the actual courses that were offered but does not exactly represent any specific 
course. The three versions of the model were based upon MLS computer labs designed for 20, 
35 and 50 stations, a range that encompasses the actual enrollment levels observed across the 
campuses. The average section size for all the classroom sections offered was 30; this is the 
enrollment value used for the classroom sections in the three cost comparisons shown below. 

The MLS version of the course requires that students have access to a multimedia laboratory 
where the courseware is available on a local area network (later versions of the MLS materials 
are available on the Internet). The capital costs of creating such a lab are detailed in the top 
panel of Table 10. These initial capital costs are converted to annual costs based upon 
estimates of useful lives of the various assets. The second panel of Table 10 shows lab monitor 
and maintenance estimates. The third panel shows the estimated annual costs for three 
different size labs. 



8 Cal State LA had experimented earlier with an MLS section without an instructor. It was 
concluded that this was not a feasible way to offer the course. 
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Table 10-Estimated Costs for Mediated Learning System Lab 



Cost items 


Initial cost 


Useful life 


Annual cost 


Capital costs : 9 








Computer workstation 


@$2,000 


4 years 


$500 


Facility remodel 


$15,000 


25 years 


$600 


Furniture 


$10,000 


1 0 years 


$1 ,000 


Value of room 


$90,000 


30 years 


$3,000 


Operating costs: 








Lab monitors 








70 hrs./wk. @$10/hr. for 30 weeks 




$21 ,000 


Lab maintenance 






$2,000 


Estimated annual costs for: 








20-station lab 






$37,600 


35-station lab 






$45,100 


50-station lab 






$52,600 



9 These capital costs are estimated as though the lab was used exclusively for the MLS courses. If 
other courses could use the lab, e.g., on a 50-50 basis, then the lab costs should be reduced 
proportionately. For MLS courses with relatively small annual enrollments, such sharing 
represents a good way to reduce the lab costs. 
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Table 1 1 provides the elements for estimating the annual direct costs of both the MLS and the 
lecture versions of the course. The site license fee is a major component of the MLS cost. Section 
staffing costs are relatively low because of the use of part-time faculty to instruct the course (the 
same staffing costs are used for both the MLS and classroom sections). The lab cost for the MLS 
course is the estimated annual capital cost from Table 10 adjusted upward as the capacity of a 
lab is reached. For the MLS course a fee, representing the net revenue from the sale of the 
workbooks (supplied as part of the site license), is treated as an offset against expenses. 

The capital cost of the classroom is based upon an estimate of $90,000 to construct and a 
30-year useful life spread over an estimated capacity of 24 sections per room per year. A 
tutor/grading cost, estimated at $ 1 7.50 per student, is added to the classroom costs because 
several of the campuses used student assistants for this purpose. 



Table 1 1 -Estimated Direct Annual Cost Elements for Remedial Mathematics 



MLS version 


Item 


Site license 


$72,500 


Section staffing 


$2,400 


Lab costs 


(see Table 1 0) 


Less revenues 


@ $35/student 


Classroom version 




Section staffing 


$2,400 


Estimated capital cost per section 


$125 


Estimated tutor/grading cost per student 


$17.50 


(1.75 hrs./student @$10 per hour) 





Finally, lab capacity must be calculated because once the capacity of a lab is reached, an 
additional lab has to be added. The calculation, shown in Table 12, is based upon the assump- 
tion that a student enrolled in an MLS section will, on average, spend five hours per week 
in the lab. The 70-hour week represents the CSU's standard classroom schedule require- 
ment of 8 a.m. - 10 p.m., five days a week. Labs could be available more than 70 hours per 
week by scheduling on weekends. (Scheduling during the summer months could also 
increase lab availability.) 

The combined cost estimates from Tables 10 and 11 are used to generate the graphs of the cost 
schedules relating estimated direct costs to annual course enrollments for both the MLS and 
lecture sections of the course. The lab capacity data in Table 12 are used to adjust the MLS cost 
upward when the capacity of a lab is reached. In actual practice the labs might be phased in (or 
shared with other courses) so that the "steps" shown in the graphs below might not have such 
sharp comers. Nevertheless, the general shape and position of the graph still provides a relatively 
good indicator for comparison purposes. 
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Table 12-Estimated MLS Lab Capacity 



Hours per week per 


Lab hours per 


Students per 


Students 
per station 


student 


week 


station per term 


per year 


5 


70 


14 


28 




Small lab 


Intermed. lab 


Large lab 


Stations per lab 


20 


35 


50 


Students per lab 


560 


980 


1,400 



The Small Lab Case illustrated in Chart Vll shows the situation where the MLS courseware is 
implemented in a lab that limits section enrollments below those of the classroom sections. 10 
In this case the MLS course costs more than the classroom course at all levels of enrollment 
even though the incremental (or marginal cost) per student enrollee is less for the MLS course 
than for the lecture course (because the $35 workbook revenues offset some of the staffing 
costs). The MLS costs are always greater than the lecture costs because the capacity of the lab is 
reached before the MLS cost graph intersects the classroom cost graph. The additional lab costs 
shift the MLS graph in such a way that the MLS costs are always above lecture costs. 



Chart Vll-Small Lab Case, MLS Section (Enrollment=20, Classroom Section Enrollment=30) 
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10 This reflects the experience of several of the campuses in the case study. It is likely the enrollment 
difference is the result of a start-up situation constrained by inadequate lab facilities rather 
than a policy decision to have lower section enrollments in the MLS course. It is a good example 
to model, however, because it illustrates an expensive implementation of the MLS technology. 
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The Intermediate Case shown in Chart VIII has the MLS lab section with an enrollment of 
30,” the same as the classroom section size. Here the MLS incremental costs per student are 
lower than in Chart VII because of the larger section size. The two graphs are converging but 
the intersection occurs at an annual enrollment level above 2,500. (If MLS section size were 
35, an intersection — a crossover or breakeven point — occurs at an enrollment of 2,000.) This 
is an important result to note because annual enrollments in the remedial mathematics courses 
average about 1,000 students across the campuses. 



Chart Vlll-lntermediate Case, MIS Section (Enrollment=30, Classroom Section Enrollment=30) 
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11 The reported average section size across all MLS sections in fall 1996 was 29. 
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Finally, the Large Lab Case in Chart IX has MLS lab sections with an enrollment of 50. Note that 
the MLS cost curve has an even flatter slope than in Chart VIII because the incremental staffing 
cost declines again as section size increases. The curves first intersect at an enrollment of 1,000. 
At about 1,400 annual enrollments the need for an additional lab causes the MLS costs to shift 
up, but after that MLS costs are less than classroom costs. 



Chart IX-Large Lab Case, MLS Section (Enrollment=50, Classroom Section Enrollment=30) 
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Summary 

Patterns emerge from the cost comparisons for the two instructional modes: 

(a) At the lower levels of annual course enrollment, the costs of the MLS version of the 
course always exceed those of the classroom version. This occurs because of the 
fixed costs associated with the MLS course including the license fee and the costs of 
establishing and operating the initial MLS lab. 

(b) The incremental (or marginal) costs of additional enrollments in the MLS course tends 
to be less than those for the regular classroom course (assuming instructor pay rates 
for the different course sections are the same). This result occurs because the MLS fee 
revenue offsets some of the course's staffing cost. Incremental costs are also reduced 
to the extent enrollment in the MLS sections is larger than in the classroom sections. 

(c) Once enrollment grows to the capacity of the MLS lab, an additional lab must be 
added causing a step-up in MLS costs. 

At the current levels of annual enrollments in remedial mathematics courses (about 1,000 
students per year per campus), the direct costs of the MLS course with section enrollments 
of 30 exceed the costs of offering the instruction to students in the classroom course with 
enrollments of 30. 

However, because the marginal costs of the MLS course tend to be less than those of 
the classroom version, with sufficient levels of annual enrollment and the appropriate 
combination of cost and fee factors, the MLS costs can be less than those of classroom 
courses. E.g., increasing MLS section size to 50 results in a crossover point at an annual 
enrollment of 1,000 students. 

These results are dependent upon all the assumptions and estimates used in the cost model. A 
sensitivity analysis focused on the MLS section with 30 students, equal to the classroom 
section size, suggests that reducing the site license fee by half shifts the MLS cost schedule so 
that the crossover point is at an annual enrollment of approximately 2,000 instead of 2,500. 

Another factor at work here is that the staffing costs for all of the remedial course sections are 
low because of the use of part-time faculty. The result is that any savings from increasing MLS 
section size are minimized because the staffing costs are small relative to the fixed costs of the 
MLS. Again, focusing upon the intermediate lab case shown in Chart VIII, holding staffing 
costs at $2,400 and increasing MLS section size to 35 results in a crossover point at an 
enrollment of 2,200. If section staffing costs are set at a higher rate (e.g., $7,750 based upon 
an average salary of $62,000 for full-time faculty and 8 courses per year), the crossover for 
MLS section size of 35 occurs at an annual course enrollment of 1,100. If MLS section size is 
increased to 50, the crossover occurs at an annual enrollment of 600. 



ERIC 



CSU Case Study 



36 



39 




U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 
Educational Resources Information Center (ERIC) 




NOTICE 

REPRODUCTION BASIS 




This document is covered by a signed “Reproduction Release 
(Blanket) form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a “Specific Document” Release form. 




This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release form 
(either “Specific Document” or “Blanket”). 




EFF-089 (9/97) 




